Teleprompting system and method

ABSTRACT

A teleprompter system and method include use of a touch-screen interface positioned intermediate to the user and a camera such that the camera captures the user&#39;s image through a transparency of the touch-screen interface. The touch screen interface is coupled to a computer and is operably connected so as to enable user control and manipulation of interactive media content generated by the computer. A video mixing component integrates images captured by the camera with interactive media content generated by the computer, as may be manipulated by the user via the touch-screen interface, to generate a coordinated presentation. The coordinated presentation can be received by one or more remote devices. The remote devices can further interact with at least the interactive media content.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 13/152,063, filed Jun. 2, 2011, which claims the benefit ofU.S. Provisional Application Ser. No. 61/350,779, filed on Jun. 2, 2010,61/370,435, filed on Aug. 3, 2010, and 61/370,453 filed on Aug. 4, 2010,the contents of which are incorporated herein by reference in theirentirety.

FIELD OF THE INVENTION

This invention relates generally to teleprompt devices, and moreparticularly to those which are used with cameras.

BACKGROUND OF THE INVENTION

Teleprompters, or prompting mechanisms associated with video and filmcameras, are well-known accessories and commonly used by actors, newsreaders, news reporters and other personnel reading off preparedinformation and documents while being shot by the camera. Theseteleprompter systems typically comprise a portion which attaches to acamera lens, including a mirror with a hood, associated with a monitorlocated at the angle thereto, so that a person standing in front of thecamera is able to view the image displayed on the monitor in the mirror.The mirror is, conventionally, a one-way mirror, thus enabling the lensbehind the mirror to capture images on the other side thereof.

The monitor is typically attached to the mirror and hood. The monitor insuch teleprompters is typically attached to a power source, as well asto a computer into which the text to be read by the reader in front ofthe camera is displayed. As the text is displayed on the monitor, it canbe read through the reflected image on the mirror mounted on the lens,and at an appropriate angle to the monitor screen. The text, whichshould be in inverted or reverse script, is scrolled up and down alongthe monitor as the reader reads through the information.

In conventional teleprompters, the mirror has on one side thereof thehood or housing which defines a chamber. The housing fits onto the endof the lens of the camera. When attached to the lens, the chamberdefined by the housing and mirror is essentially sealed to light, andthe only light permitted to enter the chamber, and hence the lens of thecamera, is that which passes through the one-way mirror. This comprisesthe image of the objects at which the lens is directed.

The users of such known teleprompters were generally limited to viewingthe passive content displayed in conjunction with teleprompter prompts,as they had no way in which they could interact with any contentso-displayed. The present invention addresses these and other needs.

Additionally, to the extent that supplemental content has been madeavailable to such viewers, this has been done through a decoupled,separate communication channel. For instance, a producer can provide aseparate communication channel with data, a video stream, or both at aURL associated with the broadcast. For example, a television station canhave on-air programming and also provide supplemental content availablethrough a website. Apart from sponsoring both sources of information,these communication channels are generally decoupled from one another.In other words, the broadcaster has only an indirect relationship to theviewer with regard to any supplemental content.

The present invention further addresses these problems in a system andmethod in which the broadcaster—who can even be an individual using aportable computer device—provides viewers with the ability to launchsupplemental content that has been curated by the broadcaster to thetopics and information sources chosen by the broadcaster. As such, amore personal and deeper experience can be had by utilizing the presentinvention.

It is with respect to these and other considerations that the disclosuremade herein is presented.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, a teleprompter systemfor generating a coordinated presentation to be viewed by one or moreremote devices is provided. The system includes a computer configured bycode and operable to generate interactive media content to be displayedon a display, a camera for capturing images of a user, and a displaycoupled to the computer that displays the media content generated fromthe computer. A touch-screen interface having a transparency ispositioned intermediate to the user and the camera and is positionedsuch that the camera captures the user's image through the transparencyof the touch-screen interface, the touch screen interface being coupledto the computer and operable to enable user control and manipulation ofthe media content generated by the computer and displayed on thedisplay. A video mixer is coupled to the camera and is operative tointegrate the user's image captured by the camera together with theinteractive media content generated by the computer as manipulated bythe user via the touch-screen interface, thereby generating thecoordinated presentation. The one or more remote devices can receive thecoordinated presentation and interact with at least the media content.

In accordance with a further aspect of the present invention, apresentation system for generating a coordinated presentation to beviewed by one or more remote devices is provided. The system includes acomputer having a processor and a memory, the computer being configuredby code stored in the memory and executed by the processor to generateinteractive media content to be displayed on a display, a camera forcapturing one or more images; and a display coupled to the computer thatdisplays the media content generated from the computer. A touch-screeninterface is coupled to the computer and operable to enable user controland manipulation of the media content generated by the computer anddisplayed on the display. A video mixer module is stored in the memorywhich, when executed by the processor, integrates the images captured bythe camera together with the interactive media content generated by thecomputer as manipulated by the user via the touch-screen interface, andthereby generates the coordinated presentation. The one or more remotedevices can receive the coordinated presentation and interact with atleast the media content.

In accordance with a further aspect of the present invention, ateleprompter system is provided, including a computer configured by codeand operable to generate media content to be displayed on a display, acamera for capturing images of a user and a display coupled to thecomputer that displays the media content generated from the computer. Atouch-screen interface having a transparency positioned intermediate tothe user and the camera is positioned such that the camera captures theuser's image through the transparency of the touch-screen interface, thetouch screen interface being coupled to the computer and operable toenable user control and manipulation of the media content generated bythe computer and displayed on the display, and a video mixer is coupledto the camera and is operative to integrate the user's image captured bythe camera together with the media content generated by the computer asmanipulated by the user via the touch-screen interface.

In accordance with a further aspect of the present invention, a one-waymirror is positioned intermediate to the camera and the touch-screeninterface. The one-way mirror is oriented such that it reflects computergenerated (“CG”) images from the display to the teleprompter userthrough the touch-screen interface and such that the camera capturesimages of the teleprompter user through the one-way mirror.

Further, optional aspects for a teleprompter system in accordance withfurther aspects of the invention include functionality provided by oneor more modules executing in a processor of a machine. For instance, agreen screen module can be provided which is operable to furtherconfigure the computer so as to render at least a portion of the contentgenerated by the computer invisible. Also, a calibration module can beprovided which is operable to further configure the computer so as tominimize a mis-registration between the user and the touch screeninterface. The calibration module can further comprise a facerecognition algorithm executing as part of the code in the computer andoperable to determine any movement of the user using the main camera oran optional further camera, with the face recognition algorithm beingfurther operable to dynamically calibrate the touch screen interface inresponse to any movement of the user. Still further modules can beprovided to provide an annotation function that imparts a visual cue tothe user on the touch screen interface, or to provide an overlayfunction to overlay the media content generated from the computer ontothe integrated images of the video mixer, or to alter the keystone ofthe media content generated from the computer and provide the alteredcontent to the overlay module for overlay onto the integrated images ofthe video mixer, or a combination of one or more of these modules.

These and other aspects, features, and advantages of the invention canbe understood with reference to the following detailed description ofcertain embodiments of the invention taken together in conjunction withthe accompanying drawing figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computer system that can be used withcertain embodiments of the invention;

FIG. 1A illustrates two sides of plane P for interacting with content inaccordance with an implementation of the invention;

FIG. 2 is a system diagram illustrating system components of oneembodiment of the invention;

FIG. 2A is a front, plan view of the touch screen interface showingcontent in relation to an on-air personality;

FIG. 2B is a rear, plan view of the touch screen interface of FIG. 2A;

FIG. 3 is a flow diagram depicting the method of certain embodiments ofthe invention;

FIG. 4 is a system diagram illustrating system components of anotherembodiment of the invention;

FIG. 4A is a flow diagram depicting one example of viewer interactionwith curated content;

FIG. 5 is a system diagram illustrating system components of stillanother embodiment of the invention;

FIG. 6 is a system diagram illustrating system components of yet anotherembodiment of the invention;

FIG. 7 a is a schematic diagram showing a desktop image;

FIG. 7 b is a schematic diagram showing a transparency map;

FIG. 8 shows a system diagram illustrating an arrangement for a rigincluding a camera, one-way mirror, touchscreen interface and monitor;

FIG. 9 shows another system diagram arrangement for a rig including acamera, two one-way mirrors, a touchscreen interface and two monitors;

FIG. 10 shows still another system diagram arrangement for a rigincluding a camera, two one-way mirrors each with an associated monitor,and a touchscreen interface;

FIG. 11 shows a system diagram and illustrates the user looking throughthe interface off-axis; and

FIG. 12 is an image of a personality (user) captured by the camerathrough the touchscreen interface, without any CG content on the screen.

WRITTEN DESCRIPTION OF CERTAIN EMBODIMENTS OF THE INVENTION

By way of overview and introduction, the present invention providesseveral embodiments of systems and methods that enable a coordinatedpresentation of a live image capture of a person (“user”) such as anon-air personality with computer-generated content with which the userinteracts during the course of the live image capture. In certainarrangements, as a departure from known teleprompter systems in whichthe on-air personality reads scripts and views content that iscontrolled off-camera by a producer and an editorial team, the user ofthe disclosed system can manipulate the computer-generated content infront of him while the audience looks on. The invention uses a videomixer and various algorithms and modules to enable this arrangement inwhich the computer-generated content is presented in a plane between theviewer and the on-air personality, rather than behind the on-airpersonality. This provides a forward-facing experience between the userand the viewers he is facing, as will be appreciated from the furtherdiscussion below.

In other arrangements, the above-referenced presentation is delivered toa remote device/player such as a television or a media center, either ofwhich has a processor, a memory, and a connection to a network throughwhich the coordinated presentation can be transmitted. As onenon-limiting example, the device/player can be a network-connectabletablet computer such as the iPad made by Apple Computer, Inc. ofCupertino, Calif. The device/player presents the images, computergenerated (“CG”) content, and potentially producer-selected content, toa viewer, and the viewer can select from among the CG content andinstantiate objects, launch applications, and otherwise interact throughthe same plane as the coordinated presentation, only from the oppositeside.

Turning briefly to FIG. 1A, the basic concept of certain arrangements ofthe present invention is schematically illustrated, upon which thefurther features and aspects described herein can be configured andimplemented in one or more physical embodiments. A user can “curate”content by selecting the content and causing its transmission to one ormore viewers, such as in connection with a live broadcast, acomputer-based multicast transmission, or a one-on-one transmissionbetween the user and the viewer. The user, such as a personality on abroadcast news or sports show, can speak to viewers and have his or herimage captured by a video camera in a conventional manner, and can alsointeract with computer-generated content presented on a touchscreeninterface 214 that is in front of him while an audience of viewers lookson using their respective players (devices) 810. A variety of modulescooperate with one another, as described more fully below, to enablethis arrangement in which the computer-generated content is presented ina plane P between the viewer and the user. This provides aforward-facing experience between the user and the viewers he is facing,as will be appreciated from the further discussion below. Moreover, theaudience of viewers can interact with the computer generated content bytouching the presentation from the other side of the plane P that liesbetween the viewer and the user (e.g., an on-air personality or anyother “curator” who has selected content for presentation to the viewerin the manner described herein). Thus, as shown in FIG. 1A, the curatoron one side of the plane P and the viewer on the other side of plane Pinteract with CG content as though they are each touching the same pointon opposites sides of a glass panel, notwithstanding the fact, ofcourse, that the curator is nowhere near the viewer and that the curatorcan be interacting with potentially a multiplicity of viewers.

The present invention is now described more fully with reference to theaccompanying drawings, in which one or more illustrated embodiments ofthe invention are shown. The invention is not limited in any way to theillustrated embodiments as the illustrated embodiments described beloware merely exemplary of the invention, which can be embodied in variousforms, as appreciated by one skilled in the art. Therefore, it is to beunderstood that any structural and functional details disclosed hereinare not to be interpreted as limiting the invention, but rather areprovided as a representative embodiment for teaching one skilled in theart one or more ways to implement the invention. Furthermore, the termsand phrases used herein are not intended to be limiting, but rather areto provide an understandable description of the invention.

It is to be appreciated that the embodiments of this invention asdiscussed below may be incorporated as a software algorithm,application, program or code residing in hardware, firmware and/or on acomputer useable medium (including software modules and browserplug-ins) that can be executed in a processor of a computer system toconfigure the processor to perform the functions described below. Such acomputer system typically includes memory storage configured to provideoutput from execution of the computer algorithm, code, or program. Anexemplary computer system is shown as a block diagram in FIG. 1depicting computer system 100. As noted above, in certain arrangements,computer system 100 is used by the curator to combine CG content with acaptured image of the user (curator), and several devices 810 are usedby viewers to view and interact with the curated content.

Although system 100 is represented herein as a standalone system, it isnot limited to such, but instead can be coupled in certain arrangementsto other computer systems via a network 830 (e.g., a local area networkor a wide area network such as the internet) or encompass otherembodiments as mentioned below. System 100 preferably includes a userinterface 105, a processor 110 (such as a digital data processor), and amemory 115. The user interface can comprise a touchscreen interface 214as shown in the arrangements of FIG. 2 and FIG. 6. Memory 115 is amemory for storing data and instructions suitable for controlling theoperation of processor 110. An implementation of memory 115 can include,for example, a random access memory (RAM), a hard drive and a read onlymemory (ROM), or any of these components. One of the components storedin memory 115 is a program 120.

Program 120 includes instructions for controlling processor 110. Program120 may be implemented as a single module or as a plurality of modulesthat operate in cooperation with one another. Program 120 iscontemplated as representing a software embodiment, or a component ormodule thereof, of the method 300 described hereinbelow.

User interface 105 includes an input device, such as a keyboard, touchscreen, tablet, or speech recognition subsystem, for enabling a user tocommunicate information and command selections to processor 110. Userinterface 105 also includes an output device such as a display or aprinter. In the case of a touch screen, the input and output functionsare provided by the same structure. A cursor control such as a mouse,track-ball, or joy stick, allows the user to manipulate a cursor on thedisplay for communicating additional information and command selectionsto processor 110. In embodiments of the present invention, portions ofthe program 120 including selected modules or code thereof can executeentirely without user input or other commands based on programmatic orautomated access to a data signal flow through other systems that may ormay not require a user interface for other reasons.

Referring now to FIGS. 1, 2, and 6, in certain arrangements userinterface 105 enables a user 250 to communicate information including CGcontent, and input commands for action by the processor 110 using thetouchscreen 214, while the user's image is captured by a camera 210. Asdescribed in U.S. Provisional Application Ser. No. 61/350,779, filed onJun. 2, 2010 and 61/370,435, filed on Aug. 3, 2010, both entitled“Teleprompting System and Method” and both incorporated herein byreference in their respective entireties, the camera 210, touchscreen214, and monitor(s) 216 are arranged relative to one or more mirrors 212to cause CG content to appear in the plane between the user 250 and thecamera 210. This arrangement is advantageous because the user 250 isfacing the camera while interacting with the content. Otherimplementations can be had in which there is no teleprompt rig 200 asshown in FIG. 6, such as using a suitably adapted application programexecuting on a more compact computer system 100. For instance, modifyingthe Facetime application available from Apple Computer, Inc. can resultin a camera facing a touch screen with which a user can interact andprovide curated content in a different setting than described as theprincipal example in this document. The key point in this regard toappreciate is that the present invention has applicability inapplications other than teleprompting, and that the embodiment of FIG. 6is merely illustrative of one application of the disclosed invention.

Part of the user interface 105 is an output device such as a display(monitor 230). Using the touchscreen interface 214, the user canmanipulate objects and cause them to be displayed on the monitor 230 orcause objects to be generated by the computer 100. Meanwhile, because ofthe one-way mirror 212, the camera only captures the user's image. Theterm “one-way mirror” refers to a mirror that is partially reflectiveand partially transparent, such as may be provided by apartially-silvered lens blank. Such a mirror exhibits the property ofallowing viewing through the mirror from a darkened side but not from abrightly lit side. Such mirrors are also referred to in the art as aone-way glass or a two-way mirror.

While program 120 is indicated as already loaded into memory 115, it maybe configured on a storage media 125 for subsequent loading into memory115. Storage media 125 can be any conventional storage media such as amagnetic tape, an optical storage media, a compact disc, or a floppydisc. Alternatively, storage media 125 can be random access memory(RAM), or other type of electronic storage, located on a remote storagesystem, such as a server that delivers the program 120 for installationand launch on a user device. As one non-limiting example, the programcan be located on a device located remotely, but on a network to whichthe system 100 is in communication.

It is to be understood that the invention is not to be limited to such acomputer system 100 as depicted in FIG. 1 but rather may be implementedon a general purpose microcomputer incorporating certain components ofsystem 100, such as one of the members of the Sun® Microsystems familyof computer systems, one of the members of the IBM® Personal Computerfamily, one of the members of the Apple® Computer family, or a myriad ofother computer processor driven systems, including: workstations,desktop computers, laptop computers, netbook computers, an iPad™ or liketablet device, a personal digital assistant (PDA), or a smart phone orother like handheld devices.

FIG. 1 further illustrates several devices 810-a, 810-b, 810-c (moregenerally, device 810) connectable, according to certain arrangements,through a network 830 so as to receive content distributed from thecomputer system 100. An arbitrary number of devices 810 can connect tothe system 100 or to other machines that distribute content output bythe system 100. Device 810-a is shown in detail, and the components ofeach device 810-b, etc. can be the same. The aforementioned iPad is oneexample of a product that is suitable as being the device 810 as itincludes the specified components for device 810-a, as shown in FIG. 1.

One distinction between the system 100 and the devices 810 is that thecomputer system 100 is configured, in certain arrangements, to permitthe user 250 to curate the content. In other words, in certainarrangements the output of the computer system 100 can include acombination of an image of the user 250, any CG content that the userhas selected for the viewer to see, and any further content that aproducer may have selected to accompany the output. Moreover, there canbe further data (e.g., one or more URLs and associated parameters) thatare output by the computer system 100, as described below. Meanwhile,the devices 810 have a program 820 that comprises code or instructionsthat are executable in the processor 110 of the device, and which may beimplemented as a single module or as a plurality of modules that operatein cooperation with one another. The program 820 enables viewing of thecombined content output by the computer system, either through a networkconnection to the computer 100 or via another server that supports oneto a multiplicity of devices 810. The program 820 further enablesinteraction by the viewer with the CG content, using the further datamentioned above, by touching and/or otherwise interacting with thecurated presentation using the user interface 105 of the device 810. Theprogram 820 is able to process any further data such as URLs that mayaccompany the output from computer system 100 and cause or enablefurther actions at the device 810. As a result, the curator/user 250 cantouch the interface 214 from one side so as to select content forconsumption by viewers, while up to a multiplicity of viewers can touchthe opposite side of the interface to instantiate, launch, or otherwiseinteract with such content. The viewer experience, therefore, integratesthe broadcaster/curator with interactive participant viewers.

FIG. 1 is intended to provide a brief, general description of anillustrative and/or suitable exemplary environment in which embodimentsof the below described present invention may be implemented. FIG. 1 isan example of a suitable environment and is not intended to suggest anylimitation as to the structure, scope of use, or functionality of anembodiment of the present invention. A particular environment should notbe interpreted as having any dependency or requirement relating to anyone or combination of components illustrated in an exemplary operatingenvironment. For example, in certain instances, one or more elements ofan environment may be deemed not necessary and omitted. In otherinstances, one or more other elements may be deemed necessary and added.

In the description that follows, certain embodiments may be describedwith reference to acts and symbolic representations of operations thatare performed by one or more computing devices, such as the computingsystem environment 100 of FIG. 1. As such, it will be understood thatsuch acts and operations, which are at times referred to as beingcomputer-executed, include the manipulation by the processor of thecomputer of electrical signals representing data in a structured form.This manipulation transforms the data or maintains them at locations inthe memory system of the computer, which reconfigures or otherwisealters the operation of the computer in a manner understood by thoseskilled in the art. The data structures in which data is maintained arephysical locations of the memory that have particular properties definedby the format of the data. However, while an embodiment is beingdescribed in the foregoing context, it is not meant to be limiting asthose of skill in the art will appreciate that the acts and operationsdescribed hereinafter may also be implemented in hardware.

Embodiments may be described in a general context of computer-executableinstructions, such as program modules, being executed by a computer.Generally, program modules include routines, programs, objects,components, data structures, etc., that perform particular tasks orimplement particular abstract data types.

With the exemplary computing system environment 100 of FIG. 1 beinggenerally shown and discussed above, the method and system of theinvention in accordance with illustrated embodiments will now bediscussed. It is to be appreciated that the method described herein hasbeen indicated in connection with a flow diagram for facilitating adescription of the principal processes of an illustrated embodiment ofthe invention; however, certain blocks can be invoked in an arbitraryorder, such as when the events drive the program flow such as in anobject-oriented program. Accordingly, the flow diagram is to beunderstood as an example flow and that the blocks can be invoked in adifferent order than as illustrated.

With further reference to FIG. 2, depicted is an embodiment of ateleprompter system according to certain arrangements, designatedgenerally by reference numeral 200 in accordance with the illustratedembodiments. System 200 includes a camera 210, such as a High Definition(HD) camera commonly used by studios for producing television media. Thecamera can take on a variety of forms and its particulars are notmaterial to the present invention; the only requirement of the camera isthat it have a signal output that can be provided to a video mixer 220,though it is desirable to have a synchronization signal included in thesignal output of the camera that can be used to synchronize the outputof the camera with any CG content (FIG. 3, step 340) and anyproducer-provided content (FIG. 3, step 345).

In the embodiment illustrated in FIG. 2, disposed between camera 210 anda teleprompter user 250 (such as a news or sports broadcaster) is aone-way mirror 212 preferably angled at an acute angle relative to theuser 250.

As one non-limiting example, in certain arrangements the one-way mirror212 can comprise a 50/50 beam splitter which transmits half of the lightthat impinges upon it from either its rear surface or its front surface,or can comprise a 80/20 beam splitter that transmits a greater portionof light through the mirror 212 (80%) than is reflected off of thesurface facing the user (20%). The transmission characteristic for themirror 212 can be selected to optimize design criteria which represent atradeoff between having computer-generated content presented to the useras a bright reflection off of the front face of the one-way mirror 212versus the camera 210 capturing a bright image of the user standingunder camera lights. Another criteria to be considered is whetheradditional one-way mirrors are utilized in the light path (as describedbelow), as such further mirrors, if utilized, reduce the amount of lighttransmission in the path.

System 200 further includes, in certain arrangements, a touch screeninterface 214 coupled to the user interface port 105 of the computersystem 100, preferably in front of the one-way mirror 212, relative tothe teleprompter user 250. A monitor 216 is preferably mounted out ofthe site-path of the camera 210, such as beneath the one-way mirror 212,above the one-way mirror 212, or to one side or another of the one-waymirror 212. The monitor 216 is coupled to computer system 100,preferably via a DVI inverter 218, as will be further explained below.

Optionally, a hood or barrier 224 can be positioned to at leastpartially block the user's view of the monitor 216 through the touchscreen 214. The barrier 224 can be an opaque element and/or a mirror.

As illustrated in FIG. 2, coupled to computer system 100 and camera 210in certain arrangements is a video mixer device 220 for mixing togetherthe video output from both the camera 210 and computer system 100, asfurther described below. The output of video mixer device 220 ispreferably coupled to a media output port 222, such as an HDMI outputport, for media distribution thereof. It should be understood that incertain arrangements, such as that depicted in FIG. 6, the functionalityof video mixer device 220 can be performed by one or more executingcomputer programs and/or applications, such as video mixer 220 a.

It is to be appreciated that in certain arrangements camera 210 ispreferably mounted on a rig mounting the one-way mirror 212, the touchscreen interface 214 and the monitor 216 such that the camera 210 ishooded adjacent the one-way mirror 212. The display 216 and the touchscreen interface 214 are spaced from one another at a distance D, andthis distance can result in mis-registration between locations on themonitor and their apparent position in the plane of the touch screeninterface. In particular, the distance D makes the monitor appearsmaller in the plane of the touchscreen, and so points on the monitorcan be offset from points on the touchscreen 214. One way of accountingfor the offset is to train the user 250 to compensate for the offset bytouching a different location on the touchscreen 214 than presented onthat screen to manually account for the offset. This technique requiressome training for the user to become accustomed to the mis-registrationissue. Another way of accounting for the offset is to use a calibrationengine as described hereinbelow. Still another way is to increase thesize of the monitor 214 relative to the touchscreen (and the mirror 212)in order to allow the monitor to fill the full expanse of thetouchscreen plane 216. An array of monitors can be used rather than onelarge one, but this solution is associated with increased cost and sizeof the rig, and is less preferred. As will be understood, a combinationof these approaches can be used in a given implementation.

The video mixing functionality can be provided by a standalone videomixer that is coupled to an output of the camera 210 and to an output ofthe computer. In the embodiment of FIG. 6, the computer system 100includes modules that implement, in code, the functionality of aninverter 218 and a video mixer 220. The inverter module is usefulbecause the images presented on the monitor 216 are typically invertedbecause they first reflect off of a one-way mirror 212 so as to appearin the proper orientation relative to the user 250. In particular, aninverter module 218 a includes code that executes in the computer 100 soas to configure the processor to re-map at least a portion of the normalvideo output of a computer in reverse and/or upside down. As such, there-mapped portion can include any CG objects created or otherwisepresented by the computer 100. Meanwhile, a teleprompt script can beretained in normal-video output orientation and served by the computer100, if present and if desired. The inverter module 218 a can includecode that implements a video driver to drive the re-mapped video signalsto the monitor 216. In this way, the inverted video output can beprovided to the monitor 216 while a normal video output is available atanother port for connection to a monitor that is viewed by persons otherthan the user 250. Alternatively, the inverter module 218 a is providedto a video processing chip of the computer system 100 as the videooutput of the computer.

The video mixer module 220 a includes code that executes in the computer100 so as to configure the processor to mix the normal video signals ofthe computer (and not the output of the inverter module 218 a) togetherwith the video output of the camera 210. The camera output is receivedat an input port of the computer 100, for example an HDMI port, and ismapped so as to retain the fidelity of the image as an array of digitalpicture elements. The output of video mixer device 220 is preferably acombination of signals from the camera 210 and the computer 100. Themanner of mixing the signals by the video mixer module 220 a to providethe combination can be varied. One algorithm for mixing the signals is abinary combination of the CG content in any graphical objects created bythe computer at a given moment in time with the mapped array from thecamera at that same moment in time. This mixing can result in the CGcontent being arranged in the foreground as an overlay in front of theimage of the user 250 captured by the camera. Such an output can befacilitated by passing the CG content through a green screen module thatachieves this binary substitution of the content of one pixel whereverthe desktop image as the target color (here, green), and then combiningthe array derived from the camera output with the filtered CG content.This green screen module, therefore, substitutes the camera image of theuser 250 wherever there is “green” on the desktop image. Anotheralgorithm for mixing the signals utilizes data in an alpha channel (an“alpha map” or “transparency map”) to obtain settings concerning thedegree of transparency, or, conversely, the opacity, of each individualCG content that is being created by the computer at a given moment intime and superimposing the array derived from the camera output with allof the computer output in accordance with the alpha map. As understoodin the art, the alpha channel provides metadata associated with the CGcontent that defines, on a grey scale, say, from 0 to 255, thetransparency or opacity that is to be assigned to a given object. Thealpha channel comprises data in the computer and associated with thevarious CG objects that are to be displayed by the computer 100 or thatis associated with the entire desktop image. As such, as shown in FIG.7A, the desktop image 700 can include several objects such as atemperature widget 710, a streamed video player 720, a document 730, anda picture 740, whereas an alpha map as shown in FIG. 7B includes atransparency value 702 for each pixel on the desktop, including values712, 722, 732, and 742 associated with respective objects 710, 720, 730,and 740. The alpha map data is accessed from a memory or other datastore and utilized by the video mixer module 220 a to transform the CGcontent into objects that can be presented over the mapped array fromthe camera, in corresponding pixel locations. The computer 100,therefore, can provide CG content with associated alpha map values tothe video mixer 22 a for combining with images provided by the camera.The combined signals from the video mixer 220 a are provided to a mediaoutput port 222, such as an HDMI output port, for media distributionthereof The media distribution can be a broadcast, multicast, or unicasttransmission of the resulting image array to an audience of one or manyviewers.

It should be understood, with regard to the embodiments of FIGS. 2 and6, that the teleprompt scripts can be projected and/or displayed ontouch screen interface 214 using one or more teleprompting applicationsand/or modules such that the user 250 can read them, as described above,however these scripts are preferably not transmitted and/or viewable byviewers of the generated content.

With the components of the present invention teleprompter system 200identified above in accordance with the illustrated embodiments of FIGS.2 and 6, their method of use will now be described with reference tomethod 300 as depicted in FIG. 3 (and with continuing reference to FIGS.2 and 6). Starting at step 310, with the teleprompter user 250preferably positioned in front of touch-screen interface 214, the camera210 captures the user's 250 image through the one-way mirror 212. Theteleprompter user 250 is preferably not able to see the camera 210 butinstead views the reflected display of the monitor 218 via the one-waymirror 212 (step 320).

The computer system 100, preferably under the control of a producer,causes images to be displayed to the teleprompter user 250, via thecomputer controlled monitor 216 and one-way mirror 212 (step 330). It isnoted in regard to the embodiment of FIGS. 2 and 6 that since theone-way mirror 212 will cause images displayed on the monitor to beinverted (that is, have the image displayed horizontally reversed and/orupside-down), the DVI inverter 218 coupled intermediate to the computersystem 100 and the monitor 216 will first invert the images output fromcomputer system 100 such that when the images are again inverted by theone-way mirror 212, they will appear in the proper orientation relativeto the teleprompter user 250. It is to be appreciated the user 250 isable to view the reflected computer generated images on the one-waymirror 212 since the touch screen interface 214 is preferably formed ofa clear (that is, substantially transparent) display.

In other arrangements, the computer system 100, preferably under thecontrol of a producer, causes images to be displayed to the teleprompteruser 250, via the computer controlled monitor 216 and one-way mirror 212(step 330). It is noted in regard to the embodiment of FIG. 6 that sincethe one-way mirror 212 will cause images displayed on the monitor to beinverted (that is, have the image displayed horizontally reversed and/orupside-down), the inverter module 218 a intercepts signals generated bythe computer system 100 representative of the desktop image and invertsthat desktop image so that when optically inverted by the one-way mirror212, the desktop image will appear in the proper orientation relative tothe teleprompter user 250. It is to be appreciated the user 250 is ableto view the reflected computer generated images on the one-way mirror212 since the touch screen interface 214 is preferably formed of a clear(that is, substantially transparent) display. The rig 200 of FIG. 6 caninclude further features such as a non-reflective coating ornon-reflective glass panel positioned to reduce any ghost images asdescribed below.

In certain arrangements, the CG images reflecting off of the mirror 212have a brightness that can reflect off of the side of the interface 214that faces the camera and create a “ghost” image of the CG image that isreversely oriented from the camera's vantage point. In order to minimizethis effect, a non-reflective coating 226 can be included on thecamera-facing side of the interface 214. Alternatively, if anon-reflective coating is desired, it can be applied to a free standing,transparent panel, such as a glass sheet 228, which is positioned in thelight path between the mirror 212 and the interface 214. If the glasspanel 228 is used, it is preferably mounted within a hood, such as canbe defined by the barrier 224, so that there is no extraneous lightbetween the glass panel 228 and the interface 214.

The touch screen interface 214 can include touch capacitive switchesthat capture a teleprompter user's 250 touch, and code such as analgorithm that executes on computer 100 so as to determine the X,Ycoordinates of the user's touch (relative to the computer controlledmonitor 216). In this way, the user can manipulate and control thedisplay of a computer 100 in the same way that a computer mouse can beused to provide a machine with coordinate information for controlling anoperation. The touch-capacitive switches can comprise a transparentconductor such as indium tin oxide which can be used to form an array oftransparent capacitive switches. In a different implementation, thetouch screen interface 214 can include an optical position sensingcircuit that operates in conjunction with code that executes in thecomputer 100 to detect and respond to the user's having interruptedorthogonal beams that pass in front of and across a plane of the touchscreen interface. For instance, infra-red light sources can bepositioned to emit beams in a plane across the surface of the touchscreen interface, and the algorithm can triangulate and determine theposition on the touch screen at which the interruption occurred in aconventional manner. Examples of such an infra-red device are the seriesof TouchKit touch control products produced by TouchKit, Inc. ofGuangzhou, China.

As shown in FIG. 2, a user interface port 105 of the computer system 100is coupled to the touch-screen interface 214, whereby the teleprompteruser's 250 hand movements on the touch-screen interface 214 controls thefunctionality and images created by or provided by the computer system100. Thus, the teleprompter user 250 is able to control and manipulatethe computer generated media content displayed by the monitor 216 asgenerated by computer system 100 by hand gesture movements imparted onthe touch-screen interface 214 (step 340). Each touch of the touchscreen interface or gesture within the plane of its sensors causes asignal 215 to be sent to the computer system 100. The signal isprocessed at step 340 so as to determine the location(s) of the touch orgesture, and the locations so-determined are then used to select contentor otherwise give effect to the user's action (e.g., the user may haveselected a hyperlink, resited a picture, moved an image, and so on).

Optionally, the user can control the transparency of a given CG objectto highlight or de-emphasize one or more objects presented on theinterface 214 that are mixed into a transmission to the viewer(s). Tocontrol the transparency, the computer 100 includes a module that tracksthe object with which the user 250 is interacting, and changes the alphamap associated with that object when the user commands such a change tothe object. For instance, the command can comprise sensing threetouchpoints on a single object followed by a rotation gesture. Arotation gesture in one direction following such a multi-touch by theuser can comprise a command to increase the transparency of an objectthat the user wishes to de-emphasize. Likewise, a rotation gesture inthe opposite direction following a multi-touch action by the user cancomprise a command to decrease the transparency (increase the opacity)of the touched-object so that the user 250 can make that object moreprominent in the transmission to the viewer. At the same time that sucha gesture is processed, the CG object on the desktop of the computer 100becomes more or less opaque, as the case may be. Meanwhile, the alphamap associated with that object has its properties modified to changethe transparency/opacity values for the object in coordination with thegesture. The video mixer 220, 220 a (discussed below) can thereaftercombine the mapped camera image and the so-modified CG object into acombined signal for distribution to viewers.

Optionally, in certain arrangements the system 200 further includes acalibration engine that executes in a processor 110 of the computersystem 100 and is operative to accommodate registration issues betweenthe display 216 and the touch screen interface 214. In particular, thedisplay 216 and the touch screen interface 214 are spaced from oneanother and that distance D (as shown in FIGS. 2 and 6) can result inthe signal processing at step 340 determining a location that isslightly different than that intended by the user, resulting in adverseconsequences such as the computer system presenting content in responseto step 340 that was not intended to be displayed. If the user is usingthe system 200 in a real-time production, this can result in greatproblems. Even if the display 216 and the touch screen interface 214 areprecisely in alignment, the effect of misregistration can result, due tothe spacing D, simply as a result of a difference in height or lateralposition of the user. To accommodate calibration issues, the calibrationengine preferably comprises a set of tools that compensate for one ormore misregistration issues. In part, the calibration engine cancomprise an algorithm, application and/or module that is executed beforethe broadcast goes live to have the user touch certain locations of thetouch screen interface 214, such as the corners, in order to account fora dominant eye that the user may have. The user is prompted to touchsuch locations, and as a result an offset is computed. The offset is ahorizontal and vertical adjustment that, when added or subtracted to acoordinate location on the interface 214, or when multiplied/divided outof the value of such coordinate location, corresponds to the samelocation on the monitor 216. Thus, the offset can be used by thecomputer to convert a touched point on the interface 214 into atranslated location that corresponds to an associated point on themonitor 216.

The calibration engine can comprise additional algorithms, applicationsand/or modules that execute in the processor of the computer system 100throughout the broadcast session that are operative to correct forheight and lateral position of the user. For example, the algorithmsutilized for this purpose can include the faceapi face-recognitionroutines available under license from Seeing Machines, Inc. of Acton,Mass. Such a face recognition routine can comprise a module thatexecutes on a processor of the computer system 100 and cooperates withone or more cameras 230 that are positioned so as to capture an image ofthe user and provide that image to the face recognition routine. Theroutine operates to identify the user's face relative to the touchscreen interface 214 and thereby provide information to ensure betterregistration between what the user selects and true locations on thedisplay 216. The module can dynamically correct for misregistration inresponse to any movement by the user throughout the broadcast session.

It is noted that the video captured by camera 210 only includes that ofthe teleprompter user 250 and does not include the aforesaid computergenerated media since the teleprompter user 250 preferably only views areflected image of the computer generated media via the one-way mirror212. In certain arrangements, the video mixer 220 receives as inputs theimages captured by camera 210, the computer-generated, user-manipulatedmedia content from computer system 100 (or a URL and parameters relatingthereto), and any further content, links and/or metadata (including URLsand parameters relating thereto) that a producer may wish to include(from step 345). These materials are integrated into a common mediaoutput file (or data stream) via a media output port, such as a HDMIoutput port 222 (step 350). In other arrangements. The video mixingmodule 220 a, however, receives as inputs the images captured by camera210 and the signals generated by the computer for presentation in thedesktop image based on the user's 250 manipulations of content, andintegrates them into a common media output file (or data stream) via amedia output port, such as a HDMI output port 222 (step 350).Preferably, a synchronization signal is in the signal output of thecamera that can be used to synchronize the output of the camera with anyCG content (step 340) and any producer-provided content (step 345).Before doing so, however, a green-screen module executing in a processor(such as within the mixer 220 or elsewhere) can be invoked to operate onat least a portion of the media content from the computer system 100(which content is to be combined in the output file or data stream so asto be in a plane that is in front of the user) so as to make thatportion invisible and thereby permit viewing of the image from thecamera 210 where that portion overlaps the camera image. As describedabove, the green-screen module can have the objects of interest in thecomputer-generated content appear within the output stream or file whilethe remainder is filtered out to reveal more of the user or the backdropof the broadcast set where the user is located. For example, the desktopimage can be set to a prescribed color (e.g., “green”), and whereverthat color appears, the mixer passes through the signal from the camera210. Thus, CG content on the desktop can be made to float in place of orin substitution for the image captured by the camera of the user 250 byemploying the green screen module.

In one implementation, the green-screen module can implement an alphamap, namely, a transparency of the overlaying image, as understood inthe art.

The media from the output port 222 is ultimately caused to be displayedon a viewer's device 810 having a display screen, such as a television,computer monitor, smart phone (e.g., iPhone), tablet device (e.g.,iPad™), and the like (step 360). In certain arrangements, the videocaptured from the camera 210 and the user manipulated computer contentare combined into a composite stream {such as by superimposing one uponthe other), via the video mixer device 220 (or video mixer module 220 a,discussed below; collectively, more generally, mixer 220). In otherarrangements, the video captured from the camera 210 and the usermanipulated computer content are combined with one another, via thevideo mixing module 220 a, and are output to the network 830.Optionally, the output can be accompanied by further data and/ormetadata such as one or more URLs and associated parameters to enablefurther functionality and guided user interaction at the viewer device810, as described below.

An artifact of the arrangement illustrated in FIG. 2 is that theaudience that views the media output as a broadcast stream or file isviewing the computer-generated, user-manipulated content from theopposite direction of the on-air personality. In other words, the on-airpersonality (the “user”) is looking at the touch screen interface 214and the computer-generated content while facing in the direction ofarrow A in FIGS. 2 and 6 whereas the audience of viewers (each one beinga “viewer”) is watching the on-air personality and seeing the computergenerated content while facing in the direction of arrow B in FIGS. 2and 6. The on-air personality is interacting with a display that is infront of him, yet the personality must interact with ahorizontally-reversed image provided by the DVI inverter 218 to ensurethat the viewing audience can perceive the computer-generated contentwith the correct orientation. The horizontally reversed image createsdifficulties for the user to interact with the content in a way that ismeaningful to the viewer because text and images that that the user seesvia the touch screen interface 214 are all reversed compared to what theviewers see. As a result, if the user points to a location on thecontent, the location that he points to does not correspond to thelocations that the viewers see, and the impression that a viewer gets isthat the on-air personality does not know where the relevant content ison the screen. Meanwhile, the display 216 provides teleprompter feeds tothe user that are in a normal orientation so that the on-air personalitycan read a script to the viewing audience, but any of thecomputer-generated content from the computer system 100 is reversed andpointing to that content can result in difficulties for the on-airpersonality. The difficulties are particularly keen when the on-airpersonality interacts with the computer-generated content using agesture such as dragging content from one position on the screen 214 toanother; the viewer sees the content in a reverse arrangement, and sothe impression to the viewer is peculiar, absent one of the solutionsdescribed next.

One non-technical solution is to train and require the user to reversehis behavior, which is difficult to do correctly or consistently.Another straightforward solution is to reverse the captured image of theon-air personality from the camera 210, so that it can be combined witha normal stream of the computer-generated content by the video mixer 220and have the images combined correctly. This solution requires that anL-R inverter circuit 218′ be inserted in the path between the camera 210and the video mixer 220 (or video mixer module 220 a). Such an invertercircuit flips the video image so that the right side of the image istransformed to the left side of the image, the left side of the image istransformed to the right side of the image, and all parts in between aresimilarly switched. There are undesirable implications here, such as thefact that any asymmetrical features of the on-air personality, includinghis or her wardrobe, will be horizontally reversed which may beunacceptable. A further issue that arises in this arrangement is thatonly a single camera can be used or else the image of the on-airpersonality will flip back and forth as cameras are changed.

To address this issue, embodiments of the invention optionally caninclude an annotation module executing in the computer system andoperative to impart, as a layer separate from the computer-generatedcontent, an indication of the relevant portion in the content to bepointed to as perceived when viewed from the viewer's vantage point. Inparticular, a producer or editor uses the annotation module to marklocations in the normal computer-generated image that are transposed asa separate layer onto the horizontally-reversed image. The annotationmodule preferably causes these marked locations to be directed to thedisplay 216 but not to the video mixer 220, and as a result, providevisual cues to the on-air personality on the touch screen display thatare not seen by any of the viewers of the broadcast. As such, theannotation module can provide marks (e.g., dots, words “touch here”,drag from here to here, etc.) that the on-air personality sees innormal, non-reversed text and can respond to so as to more gracefullymanipulate the reversed images of the computer-generated content in aninteractive manner during the broadcast. In addition, the teleprompterscript can be coordinated (synchronized) with the annotations to directthe user accordingly (“In Bagdad, armed forces [POINT TO ANNOTATION B]advanced against insurgent . . . ”).

Referring briefly to FIG. 2A, a plan view of the touch screen interface214 shows content 260 in the form of a document that the user 250 hascalled up onto the interface. The user 250 is facing into the papertoward the interface 214 and pointing to a visual cue 270 within thecontent 260. The text in the document that comprises content 260 in thisillustration is rendered backwards on in the interface 214, as indicatedby the arrow heads pointing to the left. In FIG. 2A, the user ispointing to the visual cue 270 which is on the left side of the documentimage, alongside text 272. In FIG. 2B, the viewer's perspective has theviewer seeing the on-air personality's face, and seeing the on-airpersonality pointing to the text 272. The visual cue 270 provides anassist to the user that is not seen by the viewer. The assist guides theuser to point to a desired location within the content 260 rather thanto, say, a blank region 274. Yet, if the user had pointed to thereversed text 272 in FIG. 2A, the viewer would perceive the on-airpersonality as having difficulties because the corresponding portion onthe non-reversed image is blank (region 274). The annotation modulehelps the user more graciously locate desired content within a reversedimage. Among other things, this can be helpful during real-timebroadcasts.

Optionally, a second one-way mirror 232 can be positioned behind thetouchscreen interface 214 (relative to the user 250), and between anyanti-reflection panel 228 and the touchscreen interface 214. The secondmirror 232 is parallel to the touchscreen interface 214. Because the CGcontent from the monitor 216 is bright, including any telepromptscripts, such content and scripts reflect off of the first one-waymirror 212 and pass through the second mirror 232 and are directlyviewable by the user 250. Meanwhile, the remainder of the second mirror232 is not comparably illuminated (the rig including the mirror 212,touchscreen interface 214, and monitor 216 are preferably shielded fromextraneous light) and so the CG content and script appear to floatwithin the mirror 232 as illuminated objects that are moveable,rotatable, and resizable, while the user 250 sees his/her reflectionwithin the second mirror wherever such content/scripts are not beingpresented.

Referring now to FIG. 4, system components of another embodiment of theinvention are illustrated in which like components to the system of FIG.2 have corresponding reference numbers. This arrangement has a projector416 connected to the computer system 100, optionally via DVI inverter218. The projector 416 shines on a semi-opaque layer 412. For example,the opaque layer can preferably be up to about 5% white opaque toprovide a faint image on which the computer-generated content can bepresented. (The system of FIG. 4 can be hooded so that it is dark andthe image projected by the projector 416 is more easily seen by theuser.)

The user 250 sees that content and can interact with it, all whilereceiving teleprompter guidance from a teleprompter 411. A camera 210points toward the user 250, and so does the projector 416. As such, boththe camera and the projector have the same vantage point, whichessentially requires that the user 250 interact with a reversed image.Alternatively, the computer image can be inverted (that is, horizontallyreversed), and the camera output reversed to provide the mixer with acorrect-orientation of the computer-generated output, albeit with ahorizontally-flipped image of the on-air personality (assuming that isan acceptable result). The output of the camera and the CG content fromthe computer system are combined at the video mixer 220, which candigitally combine (for example, using superimposition) a copy of thecomputer-generated content to make the content more visible than may becaptured as a reflection off of the opaque layer 412 (which has aghost-like, wispy appearance).

Referring now to FIG. 5, camera 210 is illustrated as being movablewithin a range (by the depicted arrows) and generally pointing toward atransparent LCD display 512 having a touch screen interface 214 incontact therewith. The LCD display of this embodiment requires no backlight, and instead preferably has the array of liquid crystal cells thatdefine the display mounted within a frame that displays the output ofthe computer 100, either with or without signal inversion by DVIinverter 218. As in the embodiment of FIG. 4, there are no significantcalibration issues because there is preferably no distance D between theimage or the projected image and the touch screen interface 214, 512.The image presented on the display 512 can be black and white—anartifact of present LCD displays, and if so, an overlay modulecomprising code executing in the computer can operate to overlay anin-color image that is combined with the camera output by the videomixer 220. Indeed, the overlay module can be utilized in connection withthe other embodiments described herein to overlay CG content at variousstages of the production, including within the video mixer 220, 220 a.

Optionally, a keystone module executes within the computer 100 orelsewhere in conjunction with the overlay module to correct for anyangular displacement of the camera 210 from a normal, central axisperpendicular to the display 512.

In another implementation, as shown in FIG. 6, computer system 100includes modules that implement, in code, the functionality of theinverter 218 and the mixer 220, as described in detail above.

In particular, as described above, an inverter module 218 a includescode that executes in the computer 100 so as to configure the processorto re-map at least a portion of the normal video output of a computer inreverse and/or upside down. As such, the re-mapped portion can includeany CG objects created or otherwise presented by the computer 100.Meanwhile, a teleprompt script can be retained in normal-video outputorientation and served by the computer 100, if desired. The invertermodule 218 a can include code that implements a video driver to drivethe re-mapped video signals to the monitor 216. In this way, theinverted video output can be provided to the monitor 216 while a normalvideo output is available at another port for connection to a monitorthat is viewed by persons other than the user 250. Alternatively, theinverter module 218 a is provided to a video processing chip of thecomputer system 100 as the video output of the computer. As with theinverter 218, the images presented on the monitor 216 are preferablyinverted as they reflect off of the one-way mirror 212 so as to appearin the proper orientation relative to the teleprompter user 250.

Also, a video mixer module 220 a includes code that executes in thecomputer 100 so as to configure the processor to mix the normal videooutput of the computer (and preferably not the output of the invertermodule 218 a) together with the video output of the camera 210. Asdescribed above, the camera output is received at an input port of thecomputer 100, for example an HDMI port, and is mapped so as to retainthe fidelity of the image as an array of digital picture elements. Theoutput of video mixer device 220 is a combination of signals from thecamera 210 and the computer 100. The manner of mixing the signals by thevideo mixer module 220 a to provide the combination can be varied. Onealgorithm for mixing the signals is a binary combination of the CGcontent in any graphical objects created by the computer at a givenmoment in time with the mapped array from the camera at that same momentin time. This mixing can result in the CG content being arranged in theforeground as an overlay in front of the image of the user 250 capturedby the camera. Such an output can be facilitated by passing the CGcontent through the green screen module discussed above, and thencombining the array derived from the camera output with the filtered CGcontent. Another algorithm for mixing the signals utilizes data in analpha channel (an “alpha map” or “transparency map”) to obtain settingsconcerning the degree of transparency, or, conversely, the opacity, ofeach individual CG content that is being created by the computer at agiven moment in time and combining the array derived from the cameraoutput with all of the computer output in accordance with the alpha map.As understood in the art, the alpha channel provides metadata associatedwith the CG content that defines, on a grey scale, say, from 0 to 255,the transparency or opacity that is to be assigned to a given object.The alpha channel comprises data in the computer and associated with thevarious CG objects that are to be displayed by the computer 100 orassociated with the entire desktop image. As such, as shown in FIG. 7 a,the desktop image 700 can include several objects such as a temperaturewidget 710, a streamed video player 720, a document 730, and a picture740, whereas an alpha map includes a transparency value 702 for eachpixel on the desktop, including values 712, 722, 732, and 742 associatedwith respective objects 710, 720, 730, and 740. The alpha map data isaccessed from a memory or other data store and utilized by the videomixer module 220 a to transform the CG content into objects that can bepresented over the mapped array from the camera, in corresponding pixellocations. The computer 100, therefore, can provide CG content withassociated alpha map values to the video mixer 22 a for combining withimages provided by the camera. The combined signals from the video mixer220 a are provided to a media output port 222, such as an HDMI outputport, for media distribution thereof. The media distribution can be abroadcast, multicast, or unicast transmission of the resulting imagearray to an audience of one to many viewers.

Turning briefly to FIGS. 8-10, several arrangements for a rig areillustrated. Like reference numbers are used to denote parts discussedabove, and other features such as the computer system 100 are omitted soas to focus attention on the illustrated arrangements. In FIG. 8, theangular orientation of the mirror 212 relative to the camera 210, andthe dimensions of the touchscreen interface 214 and the monitor 216 arecalled out as one exemplary arrangement, as well as the spacing to theuser's eyes 252. In FIG. 9, another arrangement for a rig has the lensof the camera 210 oriented along a central axis of the touchscreen 214,while monitors 216 a and 216 b are provided and arranged to providereflected images to the user's eye 252 through respective one-waymirrors 212 a, 212 b. Exemplary relative dimensions and spacing are alsoindicated. In FIG. 10, another rig arrangement again has the lens of thecamera 210 oriented along a central axis of the touchscreen 214, whilemonitors 216 c and 216 d are provided and arranged to provide reflectedimages to the user's eye 252 through respective one-way mirrors 212 c,212 d. Exemplary relative dimensions and spacing are also indicated.

Referring now to FIG. 11, a scenario in which the user's eyes 252 isoff-axis is illustrated. The other features of FIG. 11 are not reliedupon in this disclosure.

FIG. 12 is an image of a personality (the user 250) captured by thecamera through the touchscreen interface 214. In FIG. 12, no CG contentis on the screen.

In the various embodiments described herein, because thecomputer-generated content is presented in a plane in front of theon-air personality, there is a tension between having an image thatviewers can perceive that is not horizontally reversed and the imagethat is presented to the user who is looking back at the viewingaudience (that is, looking at the camera 210). Depending on theembodiment, there is an inversion or at least a horizontal reversaleither of the computer-generated image on the input side, or of thecamera output feed on the output side, or both. Depending on theembodiment, further modules can be used to reinforce thecomputer-generated image through superimposition, substitution, and thelike. In one embodiment, an overlay module as described above can beused to replace and/or overlay CG content in/over one or more definedregions of the desktop. In this way, even if content is presented to theuser 250 in forward orientation such that there are any ghost images inreverse text, or other artifacts due to the user interacting with thecontent, the overlay module can provide the video mixer 220, and/or thevideo mixing module 220 a with content with a normal orientation forcombination with the user image captured by the camera.

It is also to be appreciated that when the aforesaid integrated mediacontent output is provided to an Internet/network-connected interactiveuser device, such as a computer, smart phone (e.g., iPhone), tabletdevice (e.g., iPad™), or the like, the content can include, in additionto the information to be displayed and/or a composite signal output bythe computer system 100, certain metadata that can be used to enable themanipulation of the data at the viewer device 810, and/or enableselection of further content in addition to the media that has beenoutput by the media output port 222 of the computer system 100. By wayof example and not limitation, the metadata can include URLs andhyperlinks to resources available on the Internet such as one or morewebsites and the hyperlinks can be displayed for interaction with theviewer through the same presentation software that is used by theinteractive user device to display the media from the computer system100. In addition, in certain embodiments such URLs and hyperlinks canhave associated parameters that direct the viewer device 810 to specificcontent curated by the user 250 and/or his producer.

FIG. 4A depicts a flow diagram of one example 400 of viewer interactionwith curated content. The output from the media port 222 of the computersystem 100 preferably is accompanied by an image map. In oneimplementation, the output includes a client side image map. In thisimplementation, the state of the desktop image based on selections andmanipulations by the curator 250 is captured on a continuous basis (step410). Optionally, that state data is provided to a proxy server tomanage communications with plural devices 810 (step 420). Meanwhile, aclient map module executing on the device 810 captures any touch-pointson the user interface 105 of the device 810 (step 430). Suchtouch-points are conveyed through the network 830 back to the computersystem 100 or to a proxy server for action (step 440). In particular,the communications, which can be in a conventional protocol, identifythe coordinates that the user has touched, and this is compared to thestate map (step 450) and matched to the CG content that the curatorpresented to the viewer at that particular state (step 460). Thecomputer system 100 or proxy responds with either the content itself asan object, or a link to the content (step 470). The object is retrievedonto the viewer device (step 480) and is available for manipulation bythe viewer (step 490), independent of actions of any other viewer thatmay happen to be viewing the same output from the curator. In otherwords, the curator is not sharing objects on his or her desktop, butrather is enabling each viewer to select and take control of such objecton their local/remote device. Of course, if the object is proprietary, asubscription, login, enrollment, fees, or other requirements maycondition the viewer's ability to manipulate the content presented bythe curator on his or her own viewer device 810. Moreover, thecapabilities of the viewer device can be tested before providing contentto ensure compatibility with the players and other containers on theviewer device.

In another implementation, the output includes a server side image map.Similar to the implementation just described, a viewer can touch theuser interface 105 of the device 810 and the coordinates of that touchpoint are sent back to the computer system 100 or a proxy server forinterpretation. In response, the viewer device is directed to a locationwhere the content can be retrieved. This implementation reduces the loadon the computer system 100 or the proxy server, and shifts the retrievalfunctions to the viewer device.

It should be understood that the output from the computer system 100 caninclude as metadata information concerning the CG content in the form ofURLs plus parameters that enable the viewer device 810 to pull thecurated, CG content to the device for viewing. The URL can cause, forinstance, a specific item available through the network 830 (e.g., aspinning globe), while the parameters that accompany the URL enable thecurator to highlight or otherwise control the globe to indicate a regionof interest. In this mode of operation, the CG data, or at least a partof it, is an overlay on the client side, with the metadata definingwhere on the interface each object is to be presented, its size,rotational orientation, transparency, and so on. Such content can bedelivered asynchronously, permitting the user to select content and viewit long after the curator has moved on to other topics.

In one mode of operation, a producer of a show outputting the media fromthe computer system 100 selects further content and tags it to theoutput media stream to send to the viewer as an extension of thebroadcast content. For instance, the presentation software can comprisea browser program such as Safari, Internet Explorer, or Mozilla Firefoxwhich presents the media within a window thereof. The further contentcan be presented in the same window or a different window than thebrowser window that is showing the media delivered from the computersystem 100. As can be appreciated, the viewer's device can be used toselect such further content and to take one or more actions, includingwithout limitation, the downloading or streaming of further content fromthe Internet, or conduct of an e-commerce transaction.

In the event that the viewer interacts with the further content, thebroadcast media can automatically pause so that the viewer can explorethe additional content and resume viewing of the broadcast media withoutmissing anything. Optionally, the further content displayed to theviewer is filtered or otherwise tailored in view of viewer profileinformation, or demographic information, or both.

It should be appreciated that the embodiments of this invention can beincorporated as a software algorithm, program or code residing inhardware, firmware and/or on a computer useable medium (includingsoftware modules and browser plug-ins) that can be executed in aprocessor of a computer system to configure the processor to perform thefunctions that have been described. Such a computer system typicallyincludes memory storage configured to provide output from execution ofthe computer algorithm, code, or program.

It is intended that features of one embodiment can be used with equaladvantage in other embodiments described herein. As one example, thevideo mixer module 220 a implemented by code, and the manipulation ofdata and signals using one or more processors that have been configuredby such code and other modules, that transforms such data and signalsinto combined content for distribution to viewers can be used with theembodiments of FIGS. 2, 4, and 5, and in the method of FIG. 3 in thesame way as described in connection with FIG. 6. Optional embodiments ofthe invention can be understood as including the parts, elements andfeatures referred to or indicated herein, individually or collectively,in any or all combinations of two or more of the parts, elements orfeatures, and wherein specific integers are mentioned herein which haveknown equivalents in the art to which the invention relates, such knownequivalents are deemed to be incorporated herein as if individually setforth.

Although illustrated embodiments of the present invention have beendescribed, it should be understood that various changes, substitutions,and alterations can be made by one of ordinary skill in the art withoutdeparting from the scope of the present invention.

What is claimed is: 1-19. (canceled)
 20. A system comprising, at leastone processor; a memory that is accessible by the at least oneprocessor; a camera that is accessible by the at least one processor andthat is configured to capture one or more images; a display that isaccessible by the at least one processor and that is configured todisplay content; an interface accessible by the at least one processorand that is configured to receive input from a user; and an image mixermodule stored in the memory that is configured to mix graphical content;wherein, when executing at least some code stored in the memory, the atleast one processor is configured to: generate interactive media contentthat includes at least content curated by a user for interaction at oneor more remote devices; display, via the display, the generatedinteractive media content; receive, via the interface, a selection of aportion of the generated interactive media content, wherein the portionis less than all of the generated interactive media content; generate acoordinated presentation by integrating in the memory, via the imagemixer, the one or more images from the camera with the portion of thegenerated interactive media content, wherein the coordinatedpresentation is interactive at least as a function of the portion of thegenerated interactive media content; and transmit the coordinatedpresentation to the one or more remote devices.
 21. The system of claim20, wherein, when executing at least some of the code stored in thememory, the at least one processor is further configured to: receive,via the interface, a second selection of a different portion of thegenerated interactive media content, wherein the second portion is lessthan all of the generated interactive media content; generate a secondcoordinated presentation by integrating in the memory, via the imagemixer, one or more others images captured by the camera with the secondportion of the generated interactive media content, wherein the secondcoordinated presentation is interactive at least as a function of thesecond portion of the generated interactive media content; and transmitthe second coordinated presentation from the memory to the one or moreremote devices.
 22. The system of claim 21, wherein, when executing atleast some of the code stored in the memory, the at least one processoris further configured to: store the coordinated presentation and/or thesecond coordinated presentation in the memory; and provide access to thecoordinated presentation and/or the second coordinated presentation tothe one or more remote devices.
 23. The system of claim 20, wherein,when executing at least some of the code stored in the memory, the atleast one processor is further configured to: receive informationassociated with at least one interaction associated with the coordinatedpresentation from one of the one or more remote devices.
 24. The systemof claim 23, wherein, when executing at least some of the code stored inthe memory, the at least one processor is further configured to:transmit, to the one or more other remote devices, a representation ofthe received information.
 25. The system of claim 23, wherein theinformation includes a hyperlink.
 26. A system comprising, at least oneprocessor; a memory that is accessible by the at least one processor andthat stores code for configuring the at least one processor; a camerathat is accessible by the at least one processor and that is configuredto capture one or more images; a display that is accessible by the atleast one processor and that is configured to display content; aninterface accessible by the at least one processor and that isconfigured to receive input from a user; and an image mixer modulestored in the memory that is configured to mix graphical content;wherein, when executing at least some of the code stored in the memory,the at least one processor is configured to: generate interactive mediacontent that includes at least content curated by a user for interactionat one or more remote devices; display, via the display, the generatedinteractive media content; receive, via the interface, a selection of aportion of the generated interactive media content, wherein the portionis less than all of the generated interactive media content; generate acoordinated presentation by integrating in the memory, via the imagemixer, the one or more images from the camera with informationrepresenting at least one location of at least the portion of thegenerated interactive media content, wherein the coordinatedpresentation is interactive at least as a function of the informationrepresenting at least one location of at least a portion of thegenerated interactive media content; and transmit the coordinatedpresentation to the one or more remote devices.
 27. The system of claim26, wherein, when executing at least some of the code stored in thememory, the at least one processor is further configured to: receive,via the interface, a second selection of a different portion of thegenerated interactive media content, wherein the second portion is lessthan all of the generated interactive media content; generate a secondcoordinated presentation by integrating in the memory, via the imagemixer, one or more others images captured by the camera with informationrepresenting at least one location of at least the second portion of thegenerated interactive media content, wherein the second coordinatedpresentation is interactive at least as a function of the informationrepresenting at least one location of the at least second portion of thegenerated interactive media content; and transmit the second coordinatedpresentation to the one or more remote devices.
 28. The system of claim27, wherein, when executing at least some of the code stored in thememory, the at least one processor is further configured to: store thecoordinated presentation and/or the second coordinated presentation inthe memory; and provide access to the coordinated presentation and/orthe second coordinated presentation to the one or more remote devices.29. The system of claim 26, wherein, when executing at least some of thecode stored in the memory, the at least one processor is furtherconfigured to: receive interaction information associated with at leastone interaction associated with the coordinated presentation from one ofthe one or more remote devices.
 30. The system of claim 29, wherein,when executing at least some of the code stored in the memory, the atleast one processor is further configured to: transmit, to the one ormore other remote devices, a representation of the received interactioninformation.
 31. A method comprising, accessing, by at least oneprocessor, a memory that stores code for configuring the at least oneprocessor; generating, by the at least one processor, interactive mediacontent that includes at least content curated by a user for interactionat one or more remote devices; displaying, by the at least one processorvia a display that is accessible by the at least one processor, thegenerated interactive media content; receiving, via an interfaceaccessible by the at least one processor and that is configured toreceive input from a user, a selection of a portion of the generatedinteractive media content, wherein the portion is less than all of thegenerated interactive media content; generating, by the at least oneprocessor via an image mixer that is configured to mix graphical contentin the memory, a coordinated presentation by integrating in the memorythe one or more images from the camera with the portion of the generatedinteractive media content, wherein the coordinated presentation isinteractive at least as a function of the portion of the generatedinteractive media content; and transmitting the coordinated presentationto the one or more remote devices.
 32. The method of claim 31, furthercomprising: receiving, by the at least one processor via the interface,a second selection of a different portion of the generated interactivemedia content, wherein the second portion is less than all of thegenerated interactive media content; generating, by the at least oneprocessor via the image mixer, a second coordinated presentation byintegrating one or more others images captured by the camera with thesecond portion of the generated interactive media content, wherein thesecond coordinated presentation is interactive at least as a function ofthe second portion of the generated interactive media content; andtransmitting the second coordinated presentation from the memory to theone or more remote devices.
 33. The method of claim 32, furthercomprising: storing the coordinated presentation and/or the secondcoordinated presentation in the memory; and providing access to thecoordinated presentation and/or the second coordinated presentation tothe one or more remote devices.
 34. The method of claim 31, furthercomprising receiving information associated with at least oneinteraction associated with the coordinated presentation from one of theone or more remote devices.
 35. The method of claim 34, furthercomprising transmitting, to the one or more other remote devices, arepresentation of the received information.
 36. The method of claim 34,wherein the information includes a hyperlink.